A Signal-processing Framework for Image Relighting

This report reviews, in light of signal processing, the problem of relighting a Lambertian convex object with distant light source, whose crucial task is the decomposition of reflectance function into albedos (reflection coefficients) and lighting, based on a set of images and the 3-D geometry of the object from which the images were taken. A reflectance function is the result of filtering a lighting with the half-cosine kernel through a spherical convolution, that defines a linear rotationinvariant system, an extension of LTI systems in classical signal processing. This important observation maps the decomposition of reflectance function to a deconvolution, which can be facilitated in frequency-domain using spherical harmonics, an analogue of Fourier basis. As the half-cosine kernel is highly compacted in frequency-domain, reflectance functions are well approximated by a low-dimensional linear subspace spanned by the first few spherical harmonics. Therefore, the deconvolution problem can be matricized into a simple-looking matrix factorization problem. The formulation of relighting problem as a matrix factorization is carefully rederived along with discussions about spherical convolutions and spherical harmonics. Early theoretical results from the author’s previous work on solving the matrix factorization problem are also briefly reviewed. Experiments are done on synthetic data to demonstrate the use of these results.